With the recent popularity of animated GIFs on social media, there is needfor ways to index them with rich metadata. To advance research on animated GIFunderstanding, we collected a new dataset, Tumblr GIF (TGIF), with 100Kanimated GIFs from Tumblr and 120K natural language descriptions obtained viacrowdsourcing. The motivation for this work is to develop a testbed for imagesequence description systems, where the task is to generate natural languagedescriptions for animated GIFs or video clips. To ensure a high qualitydataset, we developed a series of novel quality controls to validate free-formtext input from crowdworkers. We show that there is unambiguous associationbetween visual content and natural language descriptions in our dataset, makingit an ideal benchmark for the visual content captioning task. We performextensive statistical analyses to compare our dataset to existing image andvideo description datasets. Next, we provide baseline results on the animatedGIF description task, using three representative techniques: nearest neighbor,statistical machine translation, and recurrent neural networks. Finally, weshow that models fine-tuned from our animated GIF description dataset can behelpful for automatic movie description.
展开▼
